On markov chains with sluggish transients

On sampling with Markov chains

Empirical Bayes Estimation in Nonstationary Markov chains

Estimation procedures for nonstationary Markov chains appear to be relatively sparse. This work introduces empirical  Bayes estimators  for the transition probability  matrix of a finite nonstationary  Markov chain. The data are assumed to be of  a panel study type in which each data set consists of a sequence of observations on N>=2 independent and identically dis...

On Triplet Markov Chains

The restoration of a hidden process X from an observed process Y is often performed in the framework of hidden Markov chains (HMC). HMC have been recently generalized to triplet Markov chains (TMC). In the TMC model one introduces a third random chain U and assumes that the triplet T = (X,U, Y ) is a Markov chain (MC). TMC generalize HMC but still enable the development of efficient Bayesian al...

On Markov Chains with Continuous State Space

In this expository paper, we p r o ve the following theorem, which m a y be of some use in studying Markov chain Monte Carlo methods like hit and run, the Metropolis algorithm, or the Gibbs sampler. Suppose a discrete-time Markov c hain is aperiodic, irreducible, and there is a stationary probability distribution. Then from almost all starting points the distribution of the chain at time n conv...

Parametric LTL on Markov Chains

This paper is concerned with the verification of finite Markov chains against parametrized LTL (pLTL) formulas. In pLTL, the untilmodality is equipped with a bound that contains variables; e.g., ♦6x φ asserts that φ holds within x time steps, where x is a variable on natural numbers. The central problem studied in this paper is to determine the set of parameter valuations V≺p(φ) for which the p...